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This paper describes two instructors’ efforts to more authentically engage students in a pre¬ 
service leadership program’s course called Program Planning and Evaluation by using a 
project-based learning approach. Markham, Larmer, and Ravitz (2003) describe project-based 
learning (PjBL) as “a systematic teaching method that engages students in learning knowledge 
and skills through an extended inquiry process structured around complex, authentic questions 
and carefully designed projects and tasks” (p. 4). The instructors revised the course where 
students were required to conduct a formal, standards-based evaluation of a critical issue in 
their school settings that immersed students in integrating and applying knowledge using the 
PjBL approach. The instructors used the Buck Institute for Education’s seven PjBL design 
principles for instructors (Larmer, 2015) to develop and analyze the course revisions. The 
process, challenges, and the results are discussed in this paper. 
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Introduction 


This paper examines two instructors’ efforts to use project-based learning methods in a pre¬ 
service leadership program. Markham, Larmer, and Ravitz (2003) describe project-based 
learning as “a systematic teaching method that engages students in learning knowledge and skills 
through an extended inquiry process structured around complex, authentic questions and 
carefully designed projects and tasks” (p. 4). A review of the literature on progressive 
pedagogies finds problem-based learning (PBL) and project-based learning (PjBL) are often 
discussed simultaneously. While quite similar, there are distinct differences, and for the purposes 
of this paper, it is necessary to note this difference. “The distinction between PBL and 
PjBL... according to the definition of Prince and Felder (2006) [is]...the emphasis in PjBL is on 
applying or integrating knowledge while PBL is on acquiring it” (as cited in Stefanou, Stolk, 
Prince, Chen, & Lord, 2013). The instructors revised a course where students were required to 
conduct a formal, standards-based evaluation of a critical issue in their school settings which 
immersed students in integrating and applying knowledge. Therefore, the PjBL method is a more 
logical approach. 

With the adoption of Common Core State Standards in many states, more teachers in PK- 
12 settings are adopting a PjBL model to deliver instruction. However, instructors in higher 
education are somewhat slower to embrace PjBL methods (Lee, Blackwell, Drake, & Moran, 
2014). Pre-service leadership students are constantly grappling with countless, complex 
problems in PK-12 settings and want to gain knowledge and experience in how best to resolve 
them. Leadership programs must develop authentic, real-world learning experiences for its 
emerging leaders to allow them opportunities for developing critical-thinking and problem¬ 
solving skills relevant to the work they are preparing to do in PK-12 schools. 

This paper discusses how two instructors planned and facilitated a PjBL process in a 
graduate-level course delivered in an online learning environment. The Buck Institute for 
Education’s seven PjBL design principles for instructors (Larmer, 2015) provided the frame for 
transforming a traditionally delivered course to one grounded in project methodology; they are 
(a) Design and Plan; (b) Align to Standards; (c) Build the Culture; (d) Manage Activities; (e) 
Scaffold Student Learning; (f) Assess Student Learning; and (g) Engage and Coach. The Buck 
Institutes’ following eight Gold Standards were then used to assess the implementation of the 
revised course: 

• Key Knowledge, Understandings, and Success Skills; 

• Challenging Problem or Question; 

• Sustained Inquiry; 

• Authenticity; 

• Student Voice and Choice; 

• Reflection; 

• Critique and Revision; and 

• Public Product (Larmer, Mergendoller, & Boss, 2015). 

The instructors include a reflection of the creative processes to plan and implement PjBL 
methods, the challenges that emerged with practice, and the improvements needed. 




Review of the Literature 


There is evidence project-based learning (PjBL) methods were being practiced in higher 
education in 16 th century Europe (Hugg & Wurdinger, 2007). Four centuries later, PjBL was a 
cornerstone of Dewey’s (1900) progressive education theory in early 20th century America. 
Adderley’s, (1975) description of project-based methodology is appropriate to meeting learning 
needs in our 21st century educational settings. PjBL processes involve (a) identifying a problem 
and finding a solution; (b) taking initiative in a variety of educational activities either working 
individually or in a group; (c) producing an end product; (d) engaging in the work over an 
extended period of time; and (e) teaching shifts from lecturing to facilitating the learning 
process. 

Engaging students in their learning process is challenging at all levels of education. 
Higher education is no exception and is often criticized for being out of touch with the “real” 
world (Hugg & Wurdinger, 2007). Many professors in college classrooms are familiar with a 
common complaint concerning the relevancy of a course’s content. Further compounding this 
notion is “students and educators in the 21st century are challenged by evolving employer needs, 
needs that require diverse, real experience” (Hugg & Wurdinger, 2007, p. 192). Implementing 
Dewey’s (1900) progressive pedagogies lead to a student-centered learning approach (Helle, 
Tynjala, & Olkinurora, 2006) supporting authenticity and relevancy of the learning processes. 

Stolk and Harari’s (2014) study of projects-based environments in higher education 
settings found motivation is a significant predictor of students’ engagement in high-level 
cognitions. When students are able to connect theory to practice, they are better able to 
understand the relationship between theory and their actual ‘lived’ experiences and are more 
prone to sustain motivation throughout the learning process (Brown & Freeman, 2000; Hugg & 
Wurdinger, 2007; Pintrich, 2004; Pithers, 2000; ). Cognitive benefits generated from engaged 
pedagogies (Edgerton, 2001), such as PjBL, includes deeper levels of student engagement, 
critical thinking, problem solving, reasoning, elaboration strategies, metacognition strategies, and 
skill transfer (Chick, Karis, & Kernahan, 2009; Donnelly & Fitzmaurice, 2005; Richmond & 
Hagan, 2011). 

Redesigning a Course Using Project-based Learning Methodology 

The fundamental goal of redesigning a course is to provide pre-service leadership students with a 
robust and rigorous learning experience, particularly considering the class that was chosen for 
the PjBL experiment is an online course. The course, Program Planning and Evaluation, is a 
requirement for earning the Educational Specialist degree. The instructors present the problem 
then discuss the creative process to resolve the problem using the Buck Institute for Education 
PjBL principles (Lanner, 2015) to frame the work. 

The Problem 

When the course was previously delivered, students were instructed to evaluate a program in 
their school. Oftentimes, the programs students were choosing to investigate were not ones to 
significantly contribute to the overall performance of the school. For example, some students 
would choose to evaluate a computer program their school had purchased. The evaluation results 
often lacked depth and breadth for their investigation to have meaningful effect on overall school 



improvement. The newly revised Professional Standards for Educational Leadership (PSEL), 
formerly known as Interstate School Leadership Licensure Consortium (ISLLC) standards, 
places a strong focus on developing today’s leaders with the capacity “to innovate and inspire 
staff to pursue new, creative approaches for improving schools and promoting student learning” 
(National Policy Board for Education Administration, p.l) The overall, low-level performance 
on the major assignment was an indication students lacked the kind of experiences with this 
important work to inspire and innovate. They often did not view the course relevant to their 
leadership growth. The major assignment was most often treated as simply a course requirement 
to earn a grade. The lack of depth in students’ work necessitated the shift from a traditional 
method of teaching this course to a project-based approach. 

The PjBL Method 

One of the first things the instructors addressed was to broaden the scope of the evaluation target. 
The instructors changed the focus from evaluating a program to investigating a critical issue. The 
major project of this class involved students identifying a critical issue to be addressed in their 
schools. Once students identified the critical issue they wished to explore, they conducted a 
formal, standards-based evaluation, in which, the results infonned decisions that led to better 
schooling experiences for student, families, educators, and/or communities. 

Presented below are the Buck Institute for Education’s seven principles for instructors to 
consider when designing a PjBL classroom (Larmer, 2015). The principles provided a frame for 
the creative processes in building and facilitating learning processes using PjBL methods. Each 
principle and its description is presented followed with how the instructors planned to apply the 
principle in a PjBL environment. 

Design and Plan 

Brief description. Create or adapt a project, plan it from start to finish while attending to the 
need for students to have their voice and choices throughout the project. 

Planning to apply the principle. The instructors chose to adapt the delivery of the course, 
Program Planning and Evaluation, from traditional, teacher-centered methodologies (i.e., lecture, 
textbook, exams, essays, etc.) to student-centered, project-based methodologies. This shifted the 
focus from what the instructors would be preparing and doing for each class session to what 
students would be engaging in during and in between class meetings. In the planning phase, the 
instructors focused on providing students with as much voice and choices as possible with the 
project while balancing this with keeping students moving forward in a timely manner. 

Principle 1—Align to Standards 

Brief description: Draw from and include concepts contained in subject-, disposition-, and 
skills-based standards when planning the project. 

Planning to apply the principle. Being a pre-service leadership preparation program, the course 
objectives are aligned with the ISLLC and Educational Leadership Constituent Council (ELCC) 
standards for disposition- and skills-based standards. Additionally, the focus of the course is on 



evaluating the educational programming. The subject standards of program evaluation 
established by the Joint Committee Standards for Program Evaluation (Yarbrough, Shulha, 
Hopson, & Caruthers, 2011) is essential to evaluating the critical issue. 

Principle 2—Build the Culture 

Brief description. Facilitators of PjBL develop an environment of high expectations for students 
to practice autonomy, focus on developing their own learning, seek answers/solutions to 
question/problems, practice collegiality, and produce quality work. 

Planning to apply the principle. Building a culture of high expectations is a challenge, 
particularly so in an online learning environment. The instructors used Blackboard (Bb) online 
learning systems to deliver instruction. Blackboard tools used were discussion forums, the virtual 
classroom, and group tools for planned activities which helped to create a learning community. 
Building community was a priority, therefore, the instructors set the stage with a welcome page 
containing the purpose of the course and directions to move to the Getting Started learning 
module. This module included expectations for learning, resources to begin the studies, and, 
most importantly, a link to a discussion forum called, Our Cyber Space to Get to Know One 
Another. In efforts to begin building the learning community, this forum was used for students to 
introduce themselves and to respond to and engage in conversation about themselves and what 
knowledge and skills they hoped to develop. Students engaged in a collective brainstorming 
session to compile a list of critical issues/needs in today’s schools identified as barriers to 
effective teaching and learning. The purpose of this brainstorming activity was to assist students 
in (a) generating ideas around issues to investigate in their respective schools and (b) developing 
collaborative partnerships to complete the course requirements. Next, the instructors used Bb 
Collaborate, a virtual classroom (used throughout the course for all synchronous class meetings) 
for an orientation and group discussion of the coursework, expectations, and the identified 
critical issues. These preliminary efforts helped to set a tone of high expectations, established 
collegiality among students and instructors, and fostered a shared purpose and common 
language. 

Principle 3—Manage Activities 

Brief description. Instructors in PjBL environments provide structure where students are able to 
organize and manage their work, schedules, time, and other resources, creative processes, and 
publishing/sharing their products. 

Planning to apply the principle. In addition to the typical class schedule, the instructors created 
a companion outline, the Written Report Checklist (see Appendix A). The outline was divided 
into two major tasks: Part 1: Planning Your Evaluation Design and Part 2: Conducting the 
Formal Evaluation and Producing the Report. The class schedule was developed to organize 
specific segments of the outline and the timeframes for when items were to be submitted for 
formative feedback (checkpoints). Multiple resources (university library services, government 
reports, professional web sites, etc.) were made available in Bb Learning Modules. Students 
were expected to create a formal evaluation report to share with a targeted audience who could 
include administrators, teachers, parents, or students in their schools. 



Principle 4—Scaffold Student Learning 


Brief description. Facilitators of PjBL classrooms use an assortment of instructional lessons, 
strategies, and tools to support students in meeting the goals. 

Planning to apply the principle. Scaffolding student learning has to take into consideration the 
density of course content and the various adult learning styles and needs. The instructors used 
selected Bb tools to support and maximize students’ interaction with (a) the content, (b) with 
each other, and (c) with us, the facilitators. For students to interact with content, learning 
modules were used to organize written instructions, post reading materials and resources, and 
provide links to discussion forums, and assignment tools. To support students interacting with 
each other, guided, reflective discussion forums, group tools, and the virtual classroom were 
used throughout the semester. Tools used to facilitate interaction between students and the 
instructors were the assignment tool, discussion forums, the virtual classroom, email, and phone 
calls. The instructors created an open discussion forum for students to freely communicate and 
post links to sources they found. The Written Report Checklist was used, not only as a planning 
guide for a student’s/teams’ work as they progressed through the project, but was also used to 
continually inform lesson development for the synchronous class sessions. For example, when 
the instructors moved into dense content, they acknowledged the need to use lecture-based 
teaching methods for class sessions. However, these strategies were planned to capitalize on 
‘teachable moments’ at a time it would have the most meaning for students. As the instructors 
planned, they developed steps and timelines along the way to help students move forward with 
their projects, as well as keeping the focus on the end product. 

Principle 5—Assess Student Learning 

Brief description. Facilitators of PjBL use fonnative and summative assessment in addition to 
integrating self- and/or peer-assessment of the work. 

Planning to apply the principle. Ongoing fonnative assessment of the students’/teams’ product 
in progress was provided in ‘chunks;’ that is, on the class schedule, individuals or teams would 
submit a draft of a section on the course outline for our feedback. The instructors developed a 
student survey based on the Joint Committee’s Standards for Program Evaluation (Yarbrough, 
Shulha, Hopson, & Caruthers, 2011). Students were asked to choose one of the following 
responses in regard to how they addressed the standards in their evaluation process: 1) was 
addressed, 2) partially addressed, 3) not addressed, or 4) not applicable. A rubric, based on the 
ELCC/ISLLC standards, was used for the summative assessment to provide feedback for 
disposition and skill development. Rubrics for discussion forums and for class participation, in 
addition to Bb student activity reports were used to keep students informed of their progress (See 
Appendix A). Steering papers were also provided as samples. These papers demonstrated higher- 
to lower-levels of performance. Students were encouraged to use the steering papers to self- 
assess their work and make improvements as needed. 



Principle 6—Engage and Coach 


Brief description. Facilitators of PjBL learn and create with students in addition to building 
skills, encouraging progress, praising, redirecting, and celebrating with students as needed. 

Planning to apply the principle. Because students and teams were addressing a variety of 
issues in multiple school settings, time was devoted during the virtual classroom sessions for 
engaging students and coaching purposes. Each live session started with students sharing what 
they accomplished, what they were learning, and questions that emerged. The sessions would 
end with a preview of what students would need to engage in before the next live session. 
Students served as coaches for each other in discussion forums as they shared their progress, 
problems, and findings. These experiences provided instructors and students opportunities to 
leam from one another as the evaluation of the critical issue developed over the semester. 

Using the Buck Institute for Education’s seven principles as a frame for designing a PjBL 
learning environment (Larmer, 2015) was the first step. Each instructor facilitated a section of 
the newly redesigned Program Planning and Evaluation course during the spring 2015 semester. 
The next section discusses how the PjBL model was implemented and the challenges 
encountered. 


Implementing the PjBL Method and the Emerging Challenges 

To frame our analysis and reflection of how we implemented the project-based learning 
experience, we drew from the Gold Standard PBL: Essential Project Design Elements (Lanner, 
Mergendoller, & Boss, 2015) for this discussion. There are eight design elements; each are 
presented with a brief explanation, followed with a reflection and the challenges that were 
encountered. 

Student Learning Goals 

A well-designed project requires well-designed goals. At the center of the Gold Standard PBL 
model are student learning goals organized into two parts. 

Key knowledge and understandings and key success skills. The goals for knowledge and 
understandings are developed from the subject matter’s fundamental concepts and content 
standards. The goals for success skills, also referred to as ‘21st Century Skills,’ (Lanner, 
Mergendoller, & Boss, 2015, Student Success Skills, 1) are goals to develop critical-thinking, 
problem-solving, collaboration, and self-efficacy skills, as well as discipline-specific 
professional skills. 

How the instructors addressed the design element. The purpose statement on the 
syllabus for the course served as the overarching goal for the class: 

The purpose of the course is for candidates to acquire the knowledge, skills, and 
dispositions to conceptualize, design, and implement a formal evaluation of a 
critical issue that could be impeding teaching and learning in an educational 
setting. Candidates will use appropriate qualitative and quantitative tools to gather 



data to assess the effects of the critical issue on teaching and learning and to 
inform decisions for improving practices. 

Although not perfect, the instructors consider the syllabus’ purpose statement a reflection of 
what they aspire for their students to accomplish. 

The challenge. The course objectives should be revised to address more directly key 
knowledge and understanding and key success skills. While the instructors can make an 
argument that the objectives in the syllabus contained the underlying concepts, the objectives 
should be more direct in guiding the work, feedback, and assessment processes. For example, 
most of the students’ responses to their survey results in addressing the Joint Committee’s 
Standards were not discussed in-depth in the final product. Revising the objectives to bring more 
focus to the standards will lead to better connections in the instructors’ planning, 
implementation, and assessment practices. 

Essential Project Design Elements 

In addition to the first design element above, Setting the Learning Goals, there are seven 
remaining PjBL essential design elements. They include a challenging problem or question; 
sustained inquiry; authenticity; student voice and choice; reflection; critique and revision; and a 
public product (Larmer, Mergendoller, & Boss, 2015). Following is a brief description of each 
element and how the instructors addressed it. 

Challenging problem or question. In essence, the challenging problem is what the project is all 
about. It is open-ended enough to challenge and engage students to investigate, explore, and 
search for solutions. 

How the instructors addressed the design element. Adapting the delivery to use a PjBL 
method to deliver content in the Program Planning and Evaluation course required students to 
identify a critical issue in their schools that may be having a negative impact on student 
achievement. To begin the focus on identifying a critical issue to study, students were asked to 
engage in a brainstonning activity to identify critical issues present in their schools. Focusing on 
a critical issue leads to a broader view of schooling (as opposed to focusing on a single, and most 
often a purchased, program). Students began to shift their view of school and develop more of a 
systems-theory approach during the project. This in turn helped students gain a broader sense 
that their investigation was meaningful work and could make a difference in their schools. Once 
there was agreement between the student and his/her school leader on the critical issue to be 
studied, students were asked to articulate two overarching questions that would guide and focus 
the rest of their investigation. 

The challenges. Part 1: Planning the Evaluation Design required students to design their 
evaluation. This would then serve as the blueprint for conducting the study (see Appendix A). 
The first step in the planning process was to develop the research questions, which is a complex 
exercise in and of itself. Several students found it difficult to create the overarching questions 
and continued to struggle as they progressed through the planning stage. 

Sustained inquiry. A sustained inquiry implies not only a deeper look, it also implies the 
issue will be explored over an extended period of time. When presented with a challenging 
problem or question, engaging in sustained inquiry lends itself to continual, ever deepening 
questioning and subsequently a search for potential answers from a variety of traditional and 



non-traditional sources of information. These sources are most likely field-based, action-research 
oriented, and specific to the focus of inquiry. 

How the instructors addressed the design element. As previously stated, students 
developed their overarching questions (typically two). To continue ‘drilling down’ to bring a 
narrower focus for their investigation, students also developed two sub-questions for each of 
their overarching questions. The sub-questions were then used to plan details for the 
investigation by completing infonnation in the Evaluation Design table (see Appendices A). The 
design of the evaluation included activities for obtaining the data, the data sources, data 
collection methods, who would be responsible for gathering data, how the data would be 
analyzed, and who would use the results of the investigation. 

The challenges. Not only did students struggle with developing their research questions, 
they continued to have difficulties keeping focused on their questions as they thought their way 
through each step of the Evaluation Design table (see Appendices A). For some students, there 
was a tendency for randomness across all the columns. For example, one student’s sub-question 
was seeking teachers’ perceptions of the issue being investigated, but plans for how to gather 
perceptions were not detailed in the remaining columns, clearly demonstrating a disconnect with 
what the student wanted to know and how they would get information. 

Authenticity. Students are more motivated to be fully present in the learning process when 
experiences are perceived to be relevant to their needs and to the world in which they live and 
work. Authentic projects can be conducted in real-world contexts, use actual processes, tools, or 
performance standards present in a real-world setting, have an impact on others, result in some 
thing or service benefiting others, and/or contribute to a student’s sense of personal relevancy 
when it addresses an aspect of their own identity in meaningful ways. 

How the instructors addressed the design element. Students were asked to consult with 
their colleagues and leaders in their respective schools as they contemplated the critical issue 
they wished to study. This approach not only provided students with opportunities to have a 
strong voice and meaningful choices within the project, the critical issue they collaboratively 
chose to focus on brought a much deeper sense of relevancy to the process. As students’ 
knowledge and skills grew, it only reinforced the authenticity and relevancy to their emerging 
leadership. 

The challenges. Knowing they would be sharing their final product with their targeted 
audience, students gained a much stronger sense of purpose and accountability. The instructors 
considered this a good problem to have. However, this sense of purpose and accountability 
contributed to high anxiety for some students especially as the end of the semester neared. 
Students felt they would not have the time to complete the investigation. They were right to be 
anxious about finishing a formal evaluation in a single semester, and adjustments had to be made 
to the expectations for the final product. For example, students were asked to focus their efforts 
on exploring just one of the four sub-questions rather than trying to complete all four. This 
would allow them to have the learning experience of taking one of the sub-questions through the 
entire evaluation process. Students would then have the skills needed to complete the rest of their 
evaluation plan beyond the semester. 

Student voice and choice. Providing students with opportunities to have a say and choices in the 
learning process leads to an increased sense of ownership for their personal learning and growth. 
They acquire a sense that their thoughts are valued and this can lead to students seeking higher 



levels of learning; they want to learn more. When students are fully engaged in the learning 
process, they work harder and engage in higher levels of cognition. Additionally, students will 
tend to be more fully invested to persevere and complete the final product. 

How the instructors addressed the design element. Not only were students provided a 
voice and choice in selecting their critical issue, they also were given a choice to work 
individually or in teams. They could decide what activities were needed to obtain data, who 
would be involved in collecting the data, how they would analyze and present their results, and 
decide on and suggest improvements to practices in their schools. Also, they had a voice in how 
they wanted their final product to be presented and with whom they would share the final report. 
Students were strongly encouraged to make a proposal to a state conference to share their 
investigation. 

The challenges. The instructors made a conscious decision to not use textbooks as the 
main resource in the class (two textbooks were used as references only). Because of this 
decision, the Written Report Checklist (see Appendix A) was developed. One concern during 
both the PjBL planning and implementing phases was in creating a balance between students’ 
freedom of choices and adhering to a structure for focusing the work and to make steady 
progress toward the end product. The instructors tried to leave the process open-ended enough 
where students could exercise their freedom to express their voice and choices and at the same 
time produce a final product that adhered to the Joint Committee’s Standards (Yarbrough, 
Shulha, Hopson, & Caruthers, 2011) and the components that must be present in a professional, 
formal evaluation report. 

Reflection. When one reflects, one is learning. Students and teachers should reflect continually 
on what, how, and why they are learning. Reflection can be informal and spontaneous or can be 
formalized through such processes as presenting at a conference. There are multiple ways to 
reflect: providing formative feedback, keeping learning journals, checkups and dialog in class, 
and engaging in dialog with colleagues. Reflection is a critical skill for self-efficacy (Dewey, 
1933). 

How the instructors addressed the design element. Opportunities for formal and 
informal reflection were built into the delivery of the course. The instructors began each live 
class session with a look back at what instructors and students had had done and learned since 
the previous time the class had met. In between live class meetings the discussion forums were 
used to respond to reading materials and reflect on how the infonnation informed the practices 
with learning and conducting a formal evaluation. When students posted their final product in 
their electronic portfolio, they provided a reflection of the overall process, what they learned, and 
how the experience informed their growth as a school leader. 

The challenges. This is one area the instructors believe to be a strength in the delivery of 
this course. The challenge, however, was to keep a record of reflections over time in order to 
identify trends that can be used to inform their practices and to make improvements with the 
project-based processes. 

Critique and Revision. Critical to the PjBL model is high-quality student work. To accomplish 
this goal, ongoing, constructive feedback given by both instructors and peers is necessary to 
realize high levels of work. Rubrics can be used for guiding and assessing performance, in 
addition to being used to assess overall knowledge and skill development. Fonnative feedback is 



critical for learning along the journey. Seeking outside sources to review and provide feedback 
brings real-world relevancy to the work. 

How the instructors addressed the design element. The instructors devoted scheduled 
time for students to talk about their evaluations in the live class sessions and collectively gave 
feedback. Students posted their work in both Bb assignment tool for instructor feedback and in 
the discussion forums for peer feedback. Instructors and students asked questions, encouraged 
each other, offered suggestions, and praised one another. Altogether, both students and the 
instructors collectively engaged, coached, and learned from and with one another. Steering 
papers were provided representing a highest-level score, a high-level score, a mid-level score, 
and a low-level score for students to use to gauge their level of work. Rubrics, Bb student 
activity reports, formative feedback, and summative assessments were used to support high- 
quality work. Also, the instructors encouraged students to submit proposals to state conferences 
for outside peer review of their work. 

The challenges: The tyranny of time was often a barrier for the instructors and students 
in providing the kind of detailed feedback needed for continuous development and growth. 
Students are supportive and encouraging to their peers, but they are not inclined to offer a 
critique. They left that to the instructors (understandably). 

Public product. A product can be an artifact, a decision, or a solution. A public product is a 
powerful motivating factor for producing high-quality work. The product engages key 
shareholders in meaningful discussions that help create a learning community (as opposed to a 
dialog just between an instructor and student). A public product is an effective way to 
communicate to a broader audience throughout the community. 

How the instructors addressed the design element. From the beginning of the 
instructors’ planning process with PjBL methods, the non-negotiable expectation was that the 
facilitator of the course would not be the intended audience for the final evaluation report. This 
was one of the primary purposes for why students were to identify their target audience at the 
beginning of their journey to investigate their critical issue. The facilitators’ role was to serve as 
editors and critics along the way. 

The challenges. The instructors need to create a system for follow-up checks after the 
class ends. They need to know the impact the investigation has on the practices and outcomes in 
the school. If the process continues, it is important to know how the results informed leadership 
decision-making and problem-solving skills in school settings. 

In the next section, a discussion is provided to note improvements to the process that 
emerged after implementing the redesigned course after one semester. The needs are presented 
and the actions taken to improve the processes are given. 

Improvements Needed in the PjBL Design and Delivery 

After one semester of facilitating the learning in the revised course, the instructors identified the 
following initial improvements needed with each design element: 

1. The first need was to revise the student learning objectives in the syllabus to reflect 
key knowledge and understandings and key success skills. The course syllabus 
objectives were reorganized to reflect these two key areas. This gave the instructors 
and students a better base for guiding and assessing students’ performance and work. 



Additionally, a crosswalk analysis and alignment between the objectives and the 2015 
PSEL will need to be conducted. 

2. Students needed more support with developing their overarching questions. The 
instructors provided additional resources, samples, and guidance for students before 
they submitted their overarching and sub-questions for feedback. 

3. The instructors needed to increase discussions and support to improve students’ work 
with Part 1 planning processes. Knowing that the first three design elements (aligning 
content to goals, developing challenging problems or questions, and sustained 
inquiry) needed to be well supported in order for the evaluation to maintain its 
cohesiveness throughout the process. 

4. Because the instructors had developed processes to maximize authenticity, students 
experienced added pressure to create a product that would be read by their peers and 
their identified key users. The instructors developed ongoing feedback check points 
before student submit sections for grading and before they share their work with 
school officials. 

5. The instructors made the decision not to use a traditional textbook. In its place, they 
created a prescriptive outline, the Written Report Checklist (see Appendix A). This 
prescriptive outline caused the instructors concern about the balance between 
students’ freedom of choice and adhering to a prescriptive structure. As the projects 
unfolded, the instructors gained feedback from students about the outline and the 
process. Students reported positively the Written Report Checklist kept them focused, 
helped to understand the process, and guided them in creating the final report. 

6. Reflection was crucial to the evaluation of students’ learning processes and also 
informed the changes needed in the course. The instructors collected and organized 
reflective conversations (mainly through discussion, online text chat and dialog 
during live class meetings). Collectively, these reflections helped them make data- 
informed decisions to revise the Written Report Checklist and to improve planning 
and implementation of the class. 

7. Students tended to avoid critique when giving feedback to each other. They were very 
comfortable with cheerleading, but avoided comments that would have a negative 
message. The instructors plan to coach, model, and encourage students to develop 
their skills to give professional critique for their peers in ways that support and foster 
collegiality and community. 

8. An area needing improvement is follow-up after the course to learn how the students’ 
schools used the evaluation results. Many students were not able to complete the 
evaluation as planned within the semester. Knowing this would be an ongoing 
evaluation beyond the semester, the students were encouraged to continue 
implementing their evaluation plan in other classes that require a field-based 
component. Students are asked to update instructors on their progress and how the 
schools are using the results. 


Conclusion 


The instructors have much to learn about the effects of a PjBL design on the students’ acquisition 
of knowledge and skills and the impact students’ investigations may have on their schools. 
Future leaders come to leadership programs wanting to make a difference in PK-12 settings. 



Higher education programs owe it to our emerging leadership students to develop and equip 
knowledge and skills to make a real difference in their schools. It is through PjBL methodologies 
we may better prepare emerging leaders to do well the work they aspire to accomplish. While the 
instructors engaged in this study have just started focusing on the PjBL delivery method, they 
believe they are moving in a direction to support better knowledge and skill development for pre¬ 
service leaders. 

This paper ends with two student’s final words. They were not in the same section; 
Student A. was in one facilitator’s section and Student B. was in the other section. They shared 
the following reflections: 

This has been a great experience working with B. Student. I am very thankful that you 
allowed us to do this project as a team. So far this has been my favorite assignment during this 
educational processStudent A. 

This assignment was relevant and the benchmarks assigned were great for keeping me on 
task. I appreciate [the instructor’s] willingness to continually look over my draft. This is the 
first class I have ever had where the instructor went that far above and beyond. This took the 
guess work out of the project and the worries of "Am I heading in the right direction." With that 
stress gone, I could focus more on the critical issue I was researching~ Student B. 
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Appendix A 

Written Report Checklist 


Your Program Planning and Evaluation work may be completed as either an individual 
project or as a team project. There is a great need and respect for those of you who wish to 
individually tackle/grapple with a program evaluation of a critical issue within your school 
setting for the purpose of meeting the teaching and learning needs unique to your school/district. 
Equally important is the need and respect for those of you who wish to engage in a collective 
effort to tackle, grapple, and evaluate the effects of a critical issue affecting your schools. In 
efforts to be equitable in assessing individual performances and when assessing team 
performance, we should acknowledge the expectation that a team’s performance in evaluating a 
program should include more breadth (based on the principle that “many hands make light 
work”) and more depth (from the perspective that “two heads are better than one”). 


For your consideration only (not a requirement) : If some of you wish to engage in a collective 
evaluation of a common educational program or area of focus that is present in all of your 
schools (such as educator supervision and trust; implementation of common core; the process of 
developing living, breathing school improvement planning processes, etc.), we could engage in a 
collective study. At its conclusion (which could go beyond this semester) we could consider 
submitting a proposal to one (or more) of our state educational association meetings to present 
the findings. If interested, go to the Discussion board forum called Collective Evaluation, and let 
us know and share a specific program/area of focus of interest if you have one. 


Written Report Checklist: Parts 1 & 2 
Part 1. Planning Your Evaluation Design 

First, go to Appendix A (see below), Planning for Evaluation Design Sample, replicate those 
tables, or replicate others you like better from the various samples we have explored thus far, and 
use them as your brainstonning tools to think through your evaluation design (see numbered list 
below). After you complete your tables, address the following items. You will submit your 
narrative and tables for feedback. Your design tables will be placed in your final report’s 
appendices section. You must first have your evaluation plan approved by your instructor before 
you can begin implementing your plan. 

A. Introduction 

1. Give a brief overview (2-3 paragraphs-narrative) of the critical issue you 
will be exploring, its historical impact in your school setting, and the 
purpose for evaluating the issue. 

B. Big-Picture Questions (See Appendix A, Table 1) 

1. Determine two overarching questions that you wish to “answer” with the 
results of the evaluation. 



2. Develop objectives for each question. 

C. Design of the Evaluation (See Appendix A, Table 2 headings in the top row) 

1. Evaluation Questions: Develop two questions for each of the two Big 
Picture questions (total of four questions). 

2. Activities to Observe: List the things you intend to do to get infonnation 
that can lead to an answer for the each question. 

3. Data Source: What data sources will be useful in getting the information 
on the things you want to do. 

4. Population/Sample: Report the total population size (N) and the sample 
size (n) of participants in the study for each question. 

5. Data Collection: Provide an overview of how data will be gathered and 
when. 

6. Responsibility: Detennine who will be instrumental in helping to gather 
data needed for each question. 

7. Data Analysis: Once you have data, tell how you will be analyzing the 
results. 

8. Audience-Key Users: Provide the list of key stakeholders who will be 
interested in using the results of the evaluation. (Gay, Mills, & Airasian, 
2012, p. 17; pp. 507-21) 

Part 2: Conducting the Formal Evaluation Process and Producing the Report 

Typical Evaluation Report Content 

I. Executive summary (it will be the last thing you complete) 

II. Introduction 

A. Purpose of the evaluation 

B. Key users (internal and/or external stakeholders) of the evaluation report (include 
a table similar to Appendix B, Table 1.) 

C. Limitations of the evaluation and explanation of disclaimers (Based on Joint 
Commission Standards, connected to VILA, meta-evaluation results below) 

D. Brief overview of report contents 

III. Focus of the evaluation 

A. Description of the critical issue 

B. Big Picture questions (two) 

IV. Reporting the evaluation design 

A. The leading evaluation approach (outcome-based, process-based, goals-based) 

B. Method for gathering data for each question (Quantitative, Qualitative, or Mixed- 
methods) 

C. Data elements, sources, and instruments for each question 

D. Criteria and standards used to judge the program for each question 

E. Description of data analysis strategies to detennine findings (Descriptive data: 
percentages, normal distribution of means/standard deviations, correlational 
statistics to examine relationships, etc.) 



V. Analysis of results and presentation of evaluation findings 

A. Summary of evaluation findings (use charts/figures/tables to supplement narrative 
as appropriate/needed) 

B. Interpretation of evaluation findings 

VI. Conclusions and recommendations 

A. Judgments about the program (based on each question’s criteria and standards to 
judge strengths and weaknesses) 

B. Recommendations 

VII. Appendices (requirements and examples) 

A. Required- 

1. Appendix A: Planning for Evaluation Design 

2. Appendix B: Audience-Key Users: Needs and Uses 

3. Appendix C: Standards Checklist self-evaluation instrument-completed 

B. Description (tables) of evaluation plan/design, instruments, and data analysis and 
interpretation 

C. Detailed tabulations or analyses of quantitative data and transcripts or summaries 
of qualitative data 

D. Other information as necessary (ex. a glossary/definitions/acronyms, etc.) 

(Worthen, et ah, 2010, p. 383) 
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Superintendent; 

School Board; 

High School 

Principal; 

Curriculum 

Supervisor; 

Teachers'Staff; 

Parents; 

Students 

nine weeks to 
see if 
absences 
decrease or 
increase; 
Observe 
students in 
math classes 
and in 
intervention 
class to see if 
active 

participation 

increases 

Students 

proficient 

divided by 

total tested 

and individual 

students’ 

percentage of 

growth for 

each year 

secretary will 
record absences 
of students and 
run report for 
the Curriculum 
Supervisor; 
Curriculum 
supervisor will 
observe 
students in 
math class and 
in intervention 
class 

District Test 
Coordinator 
will analyze test 
scores and 

compare the 

number of 

students and the 

percentage of 

students scoring 

proficient or 

above on the 

2012 seventh 

grade 

benchmark with 

the 2013 eighth 

grade 

benchmark to 

see if students 

showed growth 

absences will 
be analyzed at 
the end of 
each nine 
weeks grading 
period to see 
if absences 
decrease. 
Students will 
be observ ed 
in math class 
and math 
intervention 
class. 

Students 
completed the 
7* 

Augmented 
Benchmark 
Test in April 
2011 and will 
complete the 

8- grade 
Augmented 

Test in April, 

2012. Results 

will not be 

here until 

May 31, 

2012. 

Compare each 

students score 

to see if they 


N=212 

n=36 


Seventh 

Grade Math 

Augmented 

Benchmark 

Test-2011; 

Eighth Grade 

Math 

Augmented 

Benchmark 

Test-2012; 

Alg. I test — 
2012 and 

2013; 
Geometry 
test—2012 
and 2013 

Observe 
students in 
math classes 
and in 
intervention 
classes 

Analyze 
student scores 
on T 1 grade 
augmented 
math 

benchmark 
test and 
compare 
those scores 
to. student 

scores on 8 th 
grade 
augmented 
math 

benchmark 
test. Compare 
the 

percentage of 
seventh grade, 


Did the 
percentage of 
8* grade 
students 
scoring 
proficient on 
the eighth 
grade math 
benchmark 
increase 
because of the 
Math 

Intervention 
Program in 
comparison 
with math 
scores of 
students not in 
the sample 













Superintendent; 

School Board; 

High School 

Principal; 

Curriculum 

Supervisor; 

Teachers/Staff; 

Parents; 

Students 


Students 

percentage 

score on 

TARGET 

tests. 

Students’ 

mastery of 

Common 

Core 

Standards as 

tested by 

TARGET 

tests 

from seventh to 
eighth grade. 

The DTC will 
compare the 
percentage of 
students scoxine. 

fc- 

c 

above in 
seventh grade 
math, algebra I, 
and geometry in 
2012 and 2013. 

Math Teachers 
will administer 
the test, score 

the open 

response 

question, and 

record the score 

on the 

computer; 

District Test 

Coordinator 

and Curriculum 

Supervisor will 

run reports and 

compare the 

percentage 

scores on each 

of the target 

showed 
growth from 
Basic to 
Proficient to 
Advanced. 
Compare 
group 
percentage 
scores for 

2012 and 

2013 on 
seventh grade 
math, algebra 

I and 
geometry 

Students have 
completed the 
TARGET 
Pretest in 
September. 
They will 
complete the 
Interim Tests 
in October, 

January, 

March, and 

the posnest in 

May. 

Compare the 

students’ 

percentage 

score on each 

test and also 


<N 

Z cs 


TARGET test 
Pretest, 

Interim I, II, 
and III, 

Posttest 

algebra I, and 
geometry 
students 
scoring 
proficient or 
advanced in 
2012 and 
those scoring 
proficient and 
advanced in 
2013. 

Compare 
Students’ 
percentage 
score on each 
TARGET 
tests as well 
as mastery of 
common core 

standards. 

(The new 
TARGET 
PARRC 
assessment 

tests all 

common core 

standards for 
eighth grade 
math on each 

group 

(seventh 

grade. 

Algebra I, 
Geometry? 

Did the 
students show 
growth in 
individual test 
scores from 
seventh grade 
math to eighth 
grade math? 

Did student 
mastery of 8 th 
grade 

common core 

standards 
increase on 
TARGET 
tests from 
Pretest to 
Interim I to 
Interim II to 
Interim III and 
finally to the 
Posttest? 














Superintendent; 

School Board; 

High School 

Principal; 

Curriculum 

Supervisor; 

Teachers/Staff; 

Parents; 

Students 


Number of 

students 

whose grades 

in math 

showed gains 

from first nine 

weeks to 

second weeks 

to third nine 

weeks as well 

as number of 

students 

whose grades 

decline 

tests; Math 
teacher and 
curriculum 
supervisor will 
analyze the 
reports to see if 
students 
mastery of 
common core 

standards 
increased or 
decreased with 
each test 

The APSCN 
Coordinator 
will run report 
grades and list 
of grades for 

the principal 

and curriculum 

supervisor; the 

principal and 

the 

curriculum 

supervisor will 

compare grades 

from one nine 

weeks to the 

next to see if 

individual 

students 

improved their 

grade for math 

compare the 
number of 
standards 
mastered on 
each test. 

Students 
grades for 
classroom 
tests and 
assignment 
will be 
figured at the 
end of each 
nine weeks 
grading 
period (Oct., 

Jan., March., 

and May). 


N=212 

n=36 


Students’ 
Quarterly 
Report Card 
Grades and 

semester 
grades in 

Math 

Attendance 

records 

Classroom 

observations 

M 

&> 

Compare 
Students 
average 
grades in 

Math at the 
end of each 
nine weeks 
grading 
period and at 
the end of 
each semester 


Did 

Classroom 
performance 
and grade 
point average 
increase 
because of the 
Math 

Intervention 

Program? 
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